agora inbox for [email protected]
help / color / mirror / Atom feed[PATCH v8 5/7] Row pattern recognition patch (docs).
453+ messages / 4 participants
[nested] [flat]
* [PATCH v8 5/7] Row pattern recognition patch (docs).
@ 2023-09-25 05:01 Tatsuo Ishii <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Tatsuo Ishii @ 2023-09-25 05:01 UTC (permalink / raw)
---
doc/src/sgml/advanced.sgml | 52 ++++++++++++++++++++++++++++++++++
doc/src/sgml/func.sgml | 54 ++++++++++++++++++++++++++++++++++++
doc/src/sgml/ref/select.sgml | 38 +++++++++++++++++++++++--
3 files changed, 142 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/advanced.sgml b/doc/src/sgml/advanced.sgml
index 755c9f1485..f39ec8f2d5 100644
--- a/doc/src/sgml/advanced.sgml
+++ b/doc/src/sgml/advanced.sgml
@@ -537,6 +537,58 @@ WHERE pos < 3;
<literal>rank</literal> less than 3.
</para>
+ <para>
+ Row pattern common syntax can be used with row pattern common syntax to
+ perform row pattern recognition in a query. Row pattern common syntax
+ includes two sub clauses. <literal>DEFINE</literal> defines definition
+ variables along with an expression. The expression must be a logical
+ expression, which means it must
+ return <literal>TRUE</literal>, <literal>FALSE</literal>
+ or <literal>NULL</literal>. Moreover if the expression comprises a column
+ reference, it must be the argument of <function>rpr</function>. An example
+ of <literal>DEFINE</literal> is as follows.
+
+<programlisting>
+DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+</programlisting>
+
+ Note that <function>PREV</function> returns price column in the previous
+ row if it's called in a context of row pattern recognition. So in the
+ second line means the definition variable "UP" is <literal>TRUE</literal>
+ when price column in the current row is greater than the price column in
+ the previous row. Likewise, "DOWN" is <literal>TRUE</literal> when when
+ price column in the current row is lower than the price column in the
+ previous row.
+ </para>
+ <para>
+ Once <literal>DEFINE</literal> exists, <literal>PATTERN</literal> can be
+ used. <literal>PATTERN</literal> defines a sequence of rows that satisfies
+ certain conditions. For example following <literal>PATTERN</literal>
+ defines that a row starts with the condition "LOWPRICE", then one or more
+ rows satisfy "UP" and finally one or more rows satisfy "DOWN". If a
+ sequence of rows found, rpr returns the column at the starting row.
+ Example of a <literal>SELECT</literal> using the <literal>DEFINE</literal>
+ and <literal>PATTERN</literal> clause is as follows.
+
+<programlisting>
+SELECT company, tdate, price, max(price) OVER w FROM stock
+ WINDOW w AS (
+ PARTITION BY company
+ ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
+ AFTER MATCH SKIP PAST LAST ROW
+ INITIAL
+ PATTERN (LOWPRICE UP+ DOWN+)
+ DEFINE
+ LOWPRICE AS price <= 100,
+ UP AS price > PREV(price),
+ DOWN AS price < PREV(price)
+);
+</programlisting>
+ </para>
+
<para>
When a query involves multiple window functions, it is possible to write
out each one with a separate <literal>OVER</literal> clause, but this is
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
index 24ad87f910..9c99dda4ae 100644
--- a/doc/src/sgml/func.sgml
+++ b/doc/src/sgml/func.sgml
@@ -21780,6 +21780,7 @@ SELECT count(*) FROM sometable;
returns <literal>NULL</literal> if there is no such row.
</para></entry>
</row>
+
</tbody>
</tgroup>
</table>
@@ -21819,6 +21820,59 @@ SELECT count(*) FROM sometable;
Other frame specifications can be used to obtain other effects.
</para>
+ <para>
+ Row pattern recognition navigation functions are listed in
+ <xref linkend="functions-rpr-navigation-table"/>. These functions
+ can be used to describe DEFINE clause of Row pattern recognition.
+ </para>
+
+ <table id="functions-rpr-navigation-table">
+ <title>Row Pattern Navigation Functions</title>
+ <tgroup cols="1">
+ <thead>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ Function
+ </para>
+ <para>
+ Description
+ </para></entry>
+ </row>
+ </thead>
+
+ <tbody>
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>prev</primary>
+ </indexterm>
+ <function>prev</function> ( <parameter>value</parameter> <type>anyelement</type> )
+ <returnvalue>anyelement</returnvalue>
+ </para>
+ <para>
+ Returns the column value at the previous row;
+ returns NULL if there is no previous row in the window frame.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="func_table_entry"><para role="func_signature">
+ <indexterm>
+ <primary>next</primary>
+ </indexterm>
+ <function>next</function> ( <parameter>value</parameter> <type>anyelement</type> )
+ <returnvalue>anyelement</returnvalue>
+ </para>
+ <para>
+ Returns the column value at the next row;
+ returns NULL if there is no next row in the window frame.
+ </para></entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+
<note>
<para>
The SQL standard defines a <literal>RESPECT NULLS</literal> or
diff --git a/doc/src/sgml/ref/select.sgml b/doc/src/sgml/ref/select.sgml
index 0ee0cc7e64..056768b330 100644
--- a/doc/src/sgml/ref/select.sgml
+++ b/doc/src/sgml/ref/select.sgml
@@ -966,8 +966,8 @@ WINDOW <replaceable class="parameter">window_name</replaceable> AS ( <replaceabl
The <replaceable class="parameter">frame_clause</replaceable> can be one of
<synopsis>
-{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ]
-{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ]
+{ RANGE | ROWS | GROUPS } <replaceable>frame_start</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax]
+{ RANGE | ROWS | GROUPS } BETWEEN <replaceable>frame_start</replaceable> AND <replaceable>frame_end</replaceable> [ <replaceable>frame_exclusion</replaceable> ] [row_pattern_common_syntax]
</synopsis>
where <replaceable>frame_start</replaceable>
@@ -1074,6 +1074,40 @@ EXCLUDE NO OTHERS
a given peer group will be in the frame or excluded from it.
</para>
+ <para>
+ The
+ optional <replaceable class="parameter">row_pattern_common_syntax</replaceable>
+ defines the <firstterm>row pattern recognition condition</firstterm> for
+ this
+ window. <replaceable class="parameter">row_pattern_common_syntax</replaceable>
+ includes following subclauses. <literal>AFTER MATCH SKIP PAST LAST
+ ROW</literal> or <literal>AFTER MATCH SKIP TO NEXT ROW</literal> controls
+ how to proceed to next row position after a match
+ found. With <literal>AFTER MATCH SKIP PAST LAST ROW</literal> (the
+ default) next row position is next to the last row of previous match. On
+ the other hand, with <literal>AFTER MATCH SKIP TO NEXT ROW</literal> next
+ row position is always next to the last row of previous
+ match. <literal>DEFINE</literal> defines definition variables along with a
+ boolean expression. <literal>PATTERN</literal> defines a sequence of rows
+ that satisfies certain conditions using variables defined
+ in <literal>DEFINE</literal> clause. If the variable is not defined in
+ the <literal>DEFINE</literal> clause, it is implicitly assumed
+ following is defined in the <literal>DEFINE</literal> clause.
+
+<synopsis>
+<literal>variable_name</literal> AS TRUE
+</synopsis>
+
+ Note that the maximu number of variables defined
+ in <literal>DEFINE</literal> clause is 26.
+
+<synopsis>
+[ AFTER MATCH SKIP PAST LAST ROW | AFTER MATCH SKIP TO NEXT ROW ]
+PATTERN <replaceable class="parameter">pattern_variable_name</replaceable>[+] [, ...]
+DEFINE <replaceable class="parameter">definition_varible_name</replaceable> AS <replaceable class="parameter">expression</replaceable> [, ...]
+</synopsis>
+ </para>
+
<para>
The purpose of a <literal>WINDOW</literal> clause is to specify the
behavior of <firstterm>window functions</firstterm> appearing in the query's
--
2.25.1
----Next_Part(Mon_Sep_25_14_26_30_2023_752)--
Content-Type: Text/X-Patch; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="v8-0006-Row-pattern-recognition-patch-tests.patch"
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-05-04 09:32 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-05-04 09:32 UTC (permalink / raw)
Some commands that support progress reporting run sub-commands, which also
report their progress. The typical case is that REPACK builds indexes. Instead
of disabling the progress tracking of the sub-commands, we can allow both the
"parent" command and the sub-command to report their progress at the same
time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/commands/indexcmds.c | 10 ++-
src/backend/commands/repack.c | 14 +++-
src/backend/utils/activity/backend_progress.c | 76 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 9 ++-
src/include/utils/backend_status.h | 7 ++
7 files changed, 142 insertions(+), 19 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..7280e64d118 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -3918,6 +3918,12 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* more detailed comments.
*/
+ /*
+ * XXX Is there a reason not to start progress reporting here? If it's ok,
+ * then INDEX_CREATE_SUPPRESS_PROGRESS below is probably not needed.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
+
foreach(lc, indexIds)
{
char *concurrentName;
@@ -3966,8 +3972,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
if (indexRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
elog(ERROR, "cannot reindex a temporary table concurrently");
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -4144,7 +4148,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_BUILD;
progress_vals[2] = newidx->indexId;
@@ -4208,7 +4211,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* Update progress for the index to build, with the correct parent
* table involved.
*/
- pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, newidx->tableId);
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = PROGRESS_CREATEIDX_PHASE_VALIDATE_IDXSCAN;
progress_vals[2] = newidx->indexId;
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 9d162957bc3..f360f37a9da 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1937,6 +1937,7 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ reindex_params.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
@@ -3204,9 +3205,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index b0359771de5..97965a6973c 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+ Assert(beentry->st_progress_command_target == relid);
+
+ beentry->st_progress_command2 = cmdtype;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,20 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +101,20 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
if (!beentry || !pgstat_track_activities)
return;
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -124,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -144,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -155,11 +199,19 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 7a9dfa9ba3b..c2257d900af 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -334,7 +339,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
{
values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..f528f7abeec 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,13 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * subcommands are supposed to have the same target.
+ */
+ ProgressCommandType st_progress_command2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
--
2.47.3
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* [PATCH] Allow progress tracking of sub-commands.
@ 2026-06-29 11:10 Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:10 UTC (permalink / raw)
Some commands that support progress reporting run other commands which also
need to report their progress. The typical case is that REPACK ("main"
command) runs index build (sub-command).
The current approach is to disable progress tracking when a command is running
as a sub-command of another command. However, the progress reporting can be
AM-specific, so it's not really practicable to suppress the progress reporting
of the sub-commands entirely. Even if we managed to pass a flag down to all
the relevant places, we could end up with progress reporting for the
sub-command disabled in cases it's important to the user.
Instead of disabling the progress tracking of the sub-commands, this patch
allows both the main command and the sub-command to report their progress at
the same time.
---
contrib/file_fdw/file_fdw.c | 44 +++++++++++
src/backend/access/brin/brin.c | 3 +
src/backend/access/transam/xact.c | 33 +++++++-
src/backend/bootstrap/bootstrap.c | 11 ++-
src/backend/catalog/heap.c | 13 ++-
src/backend/catalog/index.c | 50 ++++++------
src/backend/catalog/toasting.c | 9 +++
src/backend/commands/indexcmds.c | 62 ++++++++-------
src/backend/commands/repack.c | 35 ++++++--
src/backend/utils/activity/backend_progress.c | 79 ++++++++++++++++---
src/backend/utils/activity/backend_status.c | 1 +
src/backend/utils/adt/pgstatfuncs.c | 15 +++-
src/include/catalog/index.h | 9 +--
src/include/utils/backend_status.h | 19 +++++
14 files changed, 294 insertions(+), 89 deletions(-)
diff --git a/contrib/file_fdw/file_fdw.c b/contrib/file_fdw/file_fdw.c
index 33a37d832ce..a60fb226320 100644
--- a/contrib/file_fdw/file_fdw.c
+++ b/contrib/file_fdw/file_fdw.c
@@ -36,6 +36,7 @@
#include "optimizer/pathnode.h"
#include "optimizer/planmain.h"
#include "optimizer/restrictinfo.h"
+#include "utils/backend_status.h"
#include "utils/acl.h"
#include "utils/memutils.h"
#include "utils/rel.h"
@@ -119,6 +120,16 @@ typedef struct FileFdwExecutionState
CopyFromState cstate; /* COPY execution state */
} FileFdwExecutionState;
+/*
+ * Since progress tracking of multiple COPY commands is not supported, the
+ * first file_fdw node of the plan needs to set pgstat_track_activities to
+ * false during startup, and the last active node needs to restore the
+ * original value during shutdown.
+ */
+static bool save_pgstat_track_activities = false;
+static int fdw_nodes = 0;
+static int active_fdw_nodes = 0;
+
/*
* SQL functions
*/
@@ -616,6 +627,12 @@ fileGetForeignPlan(PlannerInfo *root,
{
Index scan_relid = baserel->relid;
+ /*
+ * This seems to be the appropriate place to count file_fdw nodes in the
+ * plan.
+ */
+ fdw_nodes++;
+
/*
* We have no native ability to evaluate restriction clauses, so we just
* put all the scan_clauses into the plan node's qual list for the
@@ -695,6 +712,18 @@ fileBeginForeignScan(ForeignScanState *node, int eflags)
/* Add any options from the plan (currently only convert_selectively) */
options = list_concat(options, plan->fdw_private);
+ /*
+ * Save the value of pgstat_track_activities if this is the first file_fdw
+ * node of a plan containing multiple file_fdw nodes, and disable the
+ * progress tracking. The monitoring infrastructure currently does not
+ * support monitoring of multiple COPY commands.
+ */
+ if (fdw_nodes > 1 && active_fdw_nodes++ == 0)
+ {
+ save_pgstat_track_activities = pgstat_track_activities;
+ pgstat_track_activities = false;
+ }
+
/*
* Create CopyState from FDW options. We always acquire all columns, so
* as to match the expected ScanTupleSlot signature.
@@ -861,6 +890,21 @@ fileEndForeignScan(ForeignScanState *node)
festate->cstate->num_errors));
EndCopyFrom(festate->cstate);
+
+
+ /*
+ * Restore the value of pgstat_track_activities if this is the last
+ * file_fdw node of a plan containing multiple file_fdw nodes, and enable
+ * progress tracking if we disabled it earlier.
+ */
+ if (active_fdw_nodes > 0)
+ {
+ if (--active_fdw_nodes == 0)
+ {
+ pgstat_track_activities = save_pgstat_track_activities;
+ fdw_nodes = 0;
+ }
+ }
}
/*
diff --git a/src/backend/access/brin/brin.c b/src/backend/access/brin/brin.c
index bdb30752e09..f84b992478a 100644
--- a/src/backend/access/brin/brin.c
+++ b/src/backend/access/brin/brin.c
@@ -2847,8 +2847,11 @@ _brin_parallel_scan_and_build(BrinBuildState *state,
ParallelTableScanFromBrinShared(brinshared),
SO_NONE);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
reltuples = table_index_build_scan(heap, index, indexInfo, true, true,
brinbuildCallbackParallel, state, scan);
+ pgstat_progress_end_command();
/* insert the last item */
form_and_spill_tuple(state);
diff --git a/src/backend/access/transam/xact.c b/src/backend/access/transam/xact.c
index 3a89149016f..78a77fe4bb3 100644
--- a/src/backend/access/transam/xact.c
+++ b/src/backend/access/transam/xact.c
@@ -217,6 +217,8 @@ typedef struct TransactionStateData
bool parallelChildXact; /* is any parent transaction parallel? */
bool chain; /* start a new block after this one */
bool topXidLogged; /* for a subxact: is top-level XID logged? */
+ int progress_depth; /* progress tracking state before subxact
+ * started */
struct TransactionStateData *parent; /* back link to parent */
} TransactionStateData;
@@ -2253,6 +2255,12 @@ StartTransaction(void)
*/
s->state = TRANS_INPROGRESS;
+ /*
+ * Top transaction always terminates progress tracking on abort, so this
+ * field is not needed, but be tidy.
+ */
+ s->progress_depth = 0;
+
/* Schedule transaction timeout */
if (TransactionTimeout > 0)
enable_timeout_after(TRANSACTION_TIMEOUT, TransactionTimeout);
@@ -5139,6 +5147,12 @@ StartSubTransaction(void)
s->state = TRANS_INPROGRESS;
+ /*
+ * Subtransaction needs to know if it should cancel progress reporting on
+ * abort.
+ */
+ s->progress_depth = PGSTAT_PROGRESS_STATE(MyBEEntry);
+
/*
* Call start-of-subxact callbacks
*/
@@ -5298,7 +5312,24 @@ AbortSubTransaction(void)
WaitLSNCleanup();
pgstat_report_wait_end();
- pgstat_progress_end_command();
+
+ /*
+ * Abort of a subtransaction does not imply the end of monitoring for the
+ * whole transaction. Instead, we only cancel monitoring started within
+ * the subtransaction.
+ *
+ * One particular case it matters is an index function that catches an
+ * exception and lets the index build continue, so the abort should not
+ * cancel monitoring of that build.
+ *
+ * XXX Shouldn't we restore the progress state as it was before the
+ * subtransaction started? Not sure about use case - the index functions
+ * mentioned above probably don't start their own progress tracking. On
+ * the other hand, a monitored command started in a subtransaction usually
+ * does not have a parent command in the parent transaction.
+ */
+ for (int i = PGSTAT_PROGRESS_STATE(MyBEEntry); i > s->progress_depth; i--)
+ pgstat_progress_end_command();
pgaio_error_cleanup();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index b0dcd9876c5..d15a06aa99c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -40,6 +40,7 @@
#include "storage/ipc.h"
#include "storage/proc.h"
#include "storage/shmem_internal.h"
+#include "utils/backend_progress.h"
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/guc.h"
@@ -1188,7 +1189,15 @@ build_indices(void)
heap = table_open(ILHead->il_heap, NoLock);
ind = index_open(ILHead->il_ind, NoLock);
- index_build(heap, ind, ILHead->il_info, false, false, false);
+ /*
+ * Progress tracking is not really needed here, but it's simpler to
+ * start it than to disable it everywhere (including the specific
+ * AMs).
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heap));
+ index_build(heap, ind, ILHead->il_info, false, false);
+ pgstat_progress_end_command();
index_close(ind, NoLock);
table_close(heap, NoLock);
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 88087654de9..5339cfa51de 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -3588,10 +3588,15 @@ RelationTruncateIndexes(Relation heapRelation)
*/
RelationTruncate(currentIndex, 0);
- /* Initialize the index and rebuild */
- /* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, currentIndex, indexInfo, true, false,
- true);
+ /*
+ * Initialize the index and rebuild
+ *
+ * Note: we do not need to re-establish pkey setting
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(heapRelation));
+ index_build(heapRelation, currentIndex, indexInfo, true, false);
+ pgstat_progress_end_command();
/* We're done with this index */
index_close(currentIndex, NoLock);
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 9407c357f27..e5d12308b6e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -715,8 +715,6 @@ UpdateIndexRelation(Oid indexoid,
* already exists.
* INDEX_CREATE_PARTITIONED:
* create a partitioned index (table must be partitioned)
- * INDEX_CREATE_SUPPRESS_PROGRESS:
- * don't report progress during the index build.
*
* constr_flags: flags passed to index_constraint_create
* (only if INDEX_CREATE_ADD_CONSTRAINT is set)
@@ -763,7 +761,6 @@ index_create(Relation heapRelation,
bool invalid = (flags & INDEX_CREATE_INVALID) != 0;
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
- bool progress = (flags & INDEX_CREATE_SUPPRESS_PROGRESS) == 0;
char relkind;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -1280,8 +1277,7 @@ index_create(Relation heapRelation,
}
else
{
- index_build(heapRelation, indexRelation, indexInfo, false, true,
- progress);
+ index_build(heapRelation, indexRelation, indexInfo, false, true);
}
/*
@@ -1540,7 +1536,7 @@ index_concurrently_build(Oid heapRelationId,
indexInfo->ii_BrokenHotChain = false;
/* Now build the index */
- index_build(heapRel, indexRelation, indexInfo, false, true, true);
+ index_build(heapRel, indexRelation, indexInfo, false, true);
/* Roll back any GUC changes executed by index functions */
AtEOXact_GUC(false, save_nestlevel);
@@ -2221,6 +2217,8 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
*/
if (concurrent)
{
+ bool progress;
+
/*
* We must commit our transaction in order to make the first pg_index
* state update visible to other sessions. If the DROP machinery has
@@ -2291,11 +2289,12 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* to acquire an exclusive lock on our table. The lock code will
* detect deadlock and error out properly.
*
- * Note: we report progress through WaitForLockers() unconditionally
- * here, even though it will only be used when we're called by REINDEX
- * CONCURRENTLY and not when called by DROP INDEX CONCURRENTLY.
+ * Note: we report progress through WaitForLockers() only when the
+ * command is REINDEX CONCURRENTLY. There's no progress reporting for
+ * DROP INDEX CONCURRENTLY.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ progress = MyBEEntry->st_progress_command == PROGRESS_COMMAND_CREATE_INDEX;
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Updating pg_index might involve TOAST table access, so ensure we
@@ -2319,7 +2318,7 @@ index_drop(Oid indexId, bool concurrent, bool concurrent_lock_mode)
* Wait till every transaction that saw the old index state has
* finished. See above about progress reporting.
*/
- WaitForLockers(heaplocktag, AccessExclusiveLock, true);
+ WaitForLockers(heaplocktag, AccessExclusiveLock, progress);
/*
* Re-open relations to allow us to complete our actions.
@@ -3011,7 +3010,6 @@ index_update_stats(Relation rel,
*
* isreindex indicates we are recreating a previously-existing index.
* parallel indicates if parallelism may be useful.
- * progress indicates if the backend should update its progress info.
*
* Note: before Postgres 8.2, the passed-in heap and index Relations
* were automatically closed by this routine. This is no longer the case.
@@ -3022,8 +3020,7 @@ index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress)
+ bool parallel)
{
IndexBuildResult *stats;
Oid save_userid;
@@ -3074,7 +3071,6 @@ index_build(Relation heapRelation,
RestrictSearchPath();
/* Set up initial progress report status */
- if (progress)
{
const int progress_index[] = {
PROGRESS_CREATEIDX_PHASE,
@@ -3639,7 +3635,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
IndexInfo *indexInfo;
volatile bool skipped_constraint = false;
PGRUsage ru0;
- bool progress = ((params->options & REINDEXOPT_REPORT_PROGRESS) != 0);
bool set_tablespace = false;
pg_rusage_init(&ru0);
@@ -3674,7 +3669,6 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
save_nestlevel = NewGUCNestLevel();
RestrictSearchPath();
- if (progress)
{
const int progress_cols[] = {
PROGRESS_CREATEIDX_COMMAND,
@@ -3713,9 +3707,8 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
return;
}
- if (progress)
- pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
- iRel->rd_rel->relam);
+ pgstat_progress_update_param(PROGRESS_CREATEIDX_ACCESS_METHOD_OID,
+ iRel->rd_rel->relam);
/*
* If a statement is available, telling that this comes from a REINDEX
@@ -3832,7 +3825,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
/* Initialize the index and rebuild */
/* Note: we do not need to re-establish pkey setting */
- index_build(heapRelation, iRel, indexInfo, true, true, progress);
+ index_build(heapRelation, iRel, indexInfo, true, true);
/* Re-allow use of target index */
ResetReindexProcessing();
@@ -3926,8 +3919,7 @@ reindex_index(const ReindexStmt *stmt, Oid indexId,
index_close(iRel, NoLock);
table_close(heapRelation, NoLock);
- if (progress)
- pgstat_progress_end_command();
+ pgstat_progress_end_command();
}
/*
@@ -4099,9 +4091,15 @@ reindex_relation(const ReindexStmt *stmt, Oid relid, int flags,
/* Index should no longer be in the pending list */
Assert(!ReindexIsProcessingIndex(indexOid));
- /* Set index rebuild count */
- pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
- i);
+ /*
+ * Set index rebuild count - only interesting for REPACK.
+ *
+ * XXX Shouldn't we add an argument indicating that the function is
+ * being called from REPACK (or its subroutine)?
+ */
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK)
+ pgstat_progress_update_param(PROGRESS_REPACK_INDEX_REBUILD_COUNT,
+ i);
i++;
}
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index 4aa52a4bd25..672dcba84f4 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -30,6 +30,7 @@
#include "catalog/toasting.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
+#include "utils/backend_progress.h"
#include "utils/fmgroids.h"
#include "utils/rel.h"
#include "utils/syscache.h"
@@ -325,6 +326,13 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
coloptions[0] = 0;
coloptions[1] = 0;
+ /*
+ * XXX Progress reporting makes little sense for empty table, but
+ * index_create() does report the progress and it's not worth disabling it
+ * in this specific case.
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(toast_rel));
index_create(toast_rel, toast_idxname, toastIndexOid, InvalidOid,
InvalidOid, InvalidOid,
indexInfo,
@@ -333,6 +341,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
rel->rd_rel->reltablespace,
collationIds, opclassIds, NULL, coloptions, NULL, (Datum) 0,
INDEX_CREATE_IS_PRIMARY, 0, true, true, NULL);
+ pgstat_progress_end_command();
table_close(toast_rel, NoLock);
diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c
index 9ab74c8df0a..c2d4b9ff087 100644
--- a/src/backend/commands/indexcmds.c
+++ b/src/backend/commands/indexcmds.c
@@ -594,6 +594,7 @@ DefineIndex(ParseState *pstate,
Oid root_save_userid;
int root_save_sec_context;
int root_save_nestlevel;
+ bool progress_started = false;
root_save_nestlevel = NewGUCNestLevel();
@@ -622,16 +623,21 @@ DefineIndex(ParseState *pstate,
concurrent = false;
/*
- * Start progress report. If we're building a partition, this was already
- * done.
+ * Start progress report if not done yet.
+ *
+ * We could check parentIndexId and only start the reporting when
+ * processing the parent index (i.e. parentIndexId is invalid), however
+ * that wouldn't work when adding a partition to an existing table:
+ * parentIndexId is always valid in that cases.
*/
- if (!OidIsValid(parentIndexId))
+ if (MyBEEntry->st_progress_command == PROGRESS_COMMAND_INVALID)
{
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, tableId);
pgstat_progress_update_param(PROGRESS_CREATEIDX_COMMAND,
concurrent ?
PROGRESS_CREATEIDX_COMMAND_CREATE_CONCURRENTLY :
PROGRESS_CREATEIDX_COMMAND_CREATE);
+ progress_started = true;
}
/*
@@ -1277,8 +1283,7 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
- /* If this is the top-level index, we're done */
- if (!OidIsValid(parentIndexId))
+ if (progress_started)
pgstat_progress_end_command();
return address;
@@ -1585,21 +1590,18 @@ DefineIndex(ParseState *pstate,
}
}
- /*
- * Indexes on partitioned tables are not themselves built, so we're
- * done here.
- */
AtEOXact_GUC(false, root_save_nestlevel);
SetUserIdAndSecContext(root_save_userid, root_save_sec_context);
table_close(rel, NoLock);
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
{
/* Update progress for an intermediate partitioned index itself */
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
}
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1612,14 +1614,14 @@ DefineIndex(ParseState *pstate,
table_close(rel, NoLock);
/*
- * If this is the top-level index, the command is done overall;
- * otherwise, increment progress to report one child index is done.
+ * Increment progress to report one child index is done.
*/
- if (!OidIsValid(parentIndexId))
- pgstat_progress_end_command();
- else
+ if (OidIsValid(parentIndexId))
pgstat_progress_incr_param(PROGRESS_CREATEIDX_PARTITIONS_DONE, 1);
+ if (progress_started)
+ pgstat_progress_end_command();
+
return address;
}
@@ -1833,7 +1835,8 @@ DefineIndex(ParseState *pstate,
*/
UnlockRelationIdForSession(&heaprelid, ShareUpdateExclusiveLock);
- pgstat_progress_end_command();
+ if (progress_started)
+ pgstat_progress_end_command();
return address;
}
@@ -2987,7 +2990,6 @@ ReindexIndex(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
reindex_index(stmt, indOid, false, persistence, &newparams);
}
}
@@ -3110,7 +3112,6 @@ ReindexTable(const ReindexStmt *stmt, const ReindexParams *params, bool isTopLev
{
ReindexParams newparams = *params;
- newparams.options |= REINDEXOPT_REPORT_PROGRESS;
result = reindex_relation(stmt, heapOid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3535,8 +3536,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
{
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
reindex_index(stmt, relid, false, relpersistence, &newparams);
PopActiveSnapshot();
/* reindex_index() does the verbose output */
@@ -3546,8 +3546,7 @@ ReindexMultipleInternal(const ReindexStmt *stmt, const List *relids, const Reind
bool result;
ReindexParams newparams = *params;
- newparams.options |=
- REINDEXOPT_REPORT_PROGRESS | REINDEXOPT_MISSING_OK;
+ newparams.options |= REINDEXOPT_MISSING_OK;
result = reindex_relation(stmt, relid,
REINDEX_REL_PROCESS_TOAST |
REINDEX_REL_CHECK_CONSTRAINTS,
@@ -3967,7 +3966,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
elog(ERROR, "cannot reindex a temporary table concurrently");
pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, idx->tableId);
-
progress_vals[0] = PROGRESS_CREATEIDX_COMMAND_REINDEX_CONCURRENTLY;
progress_vals[1] = 0; /* initializing */
progress_vals[2] = idx->indexId;
@@ -3991,11 +3989,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Create new index definition based on given index */
newIndexId = index_create_copy(heapRel,
INDEX_CREATE_CONCURRENT |
- INDEX_CREATE_SKIP_BUILD |
- INDEX_CREATE_SUPPRESS_PROGRESS,
+ INDEX_CREATE_SKIP_BUILD,
idx->indexId,
tablespaceid,
concurrentName);
+ pgstat_progress_end_command();
/*
* Now open the relation of the new index, a session-level lock is
@@ -4114,9 +4112,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* DefineIndex() for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_1);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4153,6 +4153,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
/* Perform concurrent build of new index */
index_concurrently_build(newidx->tableId, newidx->indexId);
+ pgstat_progress_end_command();
PopActiveSnapshot();
CommitTransactionCommand();
@@ -4173,9 +4174,11 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
* for more details.
*/
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_2);
WaitForLockersMultiple(lockTags, ShareLock, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
foreach(lc, newIndexIds)
@@ -4246,6 +4249,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_3);
WaitForOlderSnapshots(limitXmin, true);
+ pgstat_progress_end_command();
CommitTransactionCommand();
}
@@ -4341,6 +4345,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
*/
INJECTION_POINT("reindex-relation-concurrently-before-set-dead", NULL);
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX, relationOid);
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_4);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
@@ -4386,6 +4391,7 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
pgstat_progress_update_param(PROGRESS_CREATEIDX_PHASE,
PROGRESS_CREATEIDX_PHASE_WAIT_5);
WaitForLockersMultiple(lockTags, AccessExclusiveLock, true);
+ pgstat_progress_end_command();
PushActiveSnapshot(GetTransactionSnapshot());
@@ -4461,8 +4467,6 @@ ReindexRelationConcurrently(const ReindexStmt *stmt, Oid relationOid, const Rein
MemoryContextDelete(private_context);
- pgstat_progress_end_command();
-
return true;
}
diff --git a/src/backend/commands/repack.c b/src/backend/commands/repack.c
index 4d177c868bb..51078d10202 100644
--- a/src/backend/commands/repack.c
+++ b/src/backend/commands/repack.c
@@ -1920,11 +1920,19 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
{
ObjectAddress object;
Oid mapped_tables[4];
+ bool is_repack;
int i;
+ /*
+ * XXX Shouldn't we add an argument indicating that the function is being
+ * called from REPACK?
+ */
+ is_repack = MyBEEntry->st_progress_command == PROGRESS_COMMAND_REPACK;
+
/* Report that we are now swapping relation files */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_SWAP_REL_FILES);
/* Zero out possible results from swapped_relation_files */
memset(mapped_tables, 0, sizeof(mapped_tables));
@@ -1981,15 +1989,17 @@ finish_heap_swap(Oid OIDOldHeap, Oid OIDNewHeap,
reindex_flags |= REINDEX_REL_FORCE_INDEXES_PERMANENT;
/* Report that we are now reindexing relations */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_REBUILD_INDEX);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_REBUILD_INDEX);
reindex_relation(NULL, OIDOldHeap, reindex_flags, &reindex_params);
}
/* Report that we are now doing clean up */
- pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
- PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
+ if (is_repack)
+ pgstat_progress_update_param(PROGRESS_REPACK_PHASE,
+ PROGRESS_REPACK_PHASE_FINAL_CLEANUP);
/*
* If the relation being rebuilt is pg_class, swap_relation_files()
@@ -3348,9 +3358,20 @@ build_new_indexes(Relation NewHeap, Relation OldHeap, List *OldIndexes)
"repacknew",
get_rel_namespace(ind->rd_index->indrelid),
false);
- newindex = index_create_copy(NewHeap, INDEX_CREATE_SUPPRESS_PROGRESS,
+
+ /*
+ * We build the index on the new heap, but after the swap phase it'll
+ * become an index on the old heap. It makes more sense to report the
+ * progress this way. (The reporting API expects that both command and
+ * subcommand deal with the same target.)
+ */
+ pgstat_progress_start_command(PROGRESS_COMMAND_CREATE_INDEX,
+ RelationGetRelid(OldHeap));
+ newindex = index_create_copy(NewHeap, 0,
oldindex, ind->rd_rel->reltablespace,
newName);
+ pgstat_progress_end_command();
+
copy_index_constraints(ind, newindex, RelationGetRelid(NewHeap));
result = lappend_oid(result, newindex);
diff --git a/src/backend/utils/activity/backend_progress.c b/src/backend/utils/activity/backend_progress.c
index 6d2049105ab..ce7ca533627 100644
--- a/src/backend/utils/activity/backend_progress.c
+++ b/src/backend/utils/activity/backend_progress.c
@@ -22,6 +22,10 @@
*
* Set st_progress_command (and st_progress_command_target) in own backend
* entry. Also, zero-initialize st_progress_param array.
+ *
+ * If command has already been started, start a sub-command. Only parameters
+ * of the sub-command are updated until pgstat_progress_end_command() is
+ * called. (Target relation must be the same for both commands.)
*-----------
*/
void
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -49,14 +74,21 @@ void
pgstat_progress_update_param(int index, int64 val)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] = val;
+ params[index] = val;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -70,14 +102,21 @@ void
pgstat_progress_incr_param(int index, int64 incr)
{
volatile PgBackendStatus *beentry = MyBEEntry;
+ volatile int64 *params;
Assert(index >= 0 && index < PGSTAT_NUM_PROGRESS_PARAM);
if (!beentry || !pgstat_track_activities)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_param[index] += incr;
+ params[index] += incr;
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
@@ -122,17 +161,24 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
{
volatile PgBackendStatus *beentry = MyBEEntry;
int i;
+ volatile int64 *params;
if (!beentry || !pgstat_track_activities || nparam == 0)
return;
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID ||
+ beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID);
+
+ params = (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID) ?
+ beentry->st_progress_param : beentry->st_progress_param2;
+
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
for (i = 0; i < nparam; ++i)
{
Assert(index[i] >= 0 && index[i] < PGSTAT_NUM_PROGRESS_PARAM);
- beentry->st_progress_param[index[i]] = val[i];
+ params[index[i]] = val[i];
}
PGSTAT_END_WRITE_ACTIVITY(beentry);
@@ -142,7 +188,7 @@ pgstat_progress_update_multi_param(int nparam, const int *index,
* pgstat_progress_end_command() -
*
* Reset st_progress_command (and st_progress_command_target) in own backend
- * entry. This signals the end of the command.
+ * entry. This signals the end of the command (or a sub-command).
*-----------
*/
void
@@ -153,11 +199,20 @@ pgstat_progress_end_command(void)
if (!beentry || !pgstat_track_activities)
return;
- if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
- return;
-
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
- beentry->st_progress_command_target = InvalidOid;
+
+ if (beentry->st_progress_command2 != PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target2 = InvalidOid;
+ }
+ else
+ {
+ beentry->st_progress_command = PROGRESS_COMMAND_INVALID;
+ beentry->st_progress_command_target = InvalidOid;
+ }
+
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
diff --git a/src/backend/utils/activity/backend_status.c b/src/backend/utils/activity/backend_status.c
index d685fc5cd87..15675992415 100644
--- a/src/backend/utils/activity/backend_status.c
+++ b/src/backend/utils/activity/backend_status.c
@@ -284,6 +284,7 @@ pgstat_bestart_initial(void)
lbeentry.st_state = STATE_STARTING;
lbeentry.st_progress_command = PROGRESS_COMMAND_INVALID;
+ lbeentry.st_progress_command2 = PROGRESS_COMMAND_INVALID;
lbeentry.st_progress_command_target = InvalidOid;
lbeentry.st_query_id = INT64CONST(0);
lbeentry.st_plan_id = INT64CONST(0);
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 6f9c9c72de5..d0513d6931b 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -314,6 +314,7 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
Datum values[PG_STAT_GET_PROGRESS_COLS] = {0};
bool nulls[PG_STAT_GET_PROGRESS_COLS] = {0};
int i;
+ volatile int64 *params;
local_beentry = pgstat_get_local_beentry_by_index(curr_backend);
beentry = &local_beentry->backendStatus;
@@ -322,7 +323,11 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
* Report values for only those backends which are running the given
* command.
*/
- if (beentry->st_progress_command != cmdtype)
+ if (beentry->st_progress_command == cmdtype)
+ params = beentry->st_progress_param;
+ else if (beentry->st_progress_command2 == cmdtype)
+ params = beentry->st_progress_param2;
+ else
continue;
/* Value available to all callers */
@@ -332,9 +337,13 @@ pg_stat_get_progress_info(PG_FUNCTION_ARGS)
/* show rest of the values including relid only to role members */
if (HAS_PGSTAT_PERMISSIONS(beentry->st_userid))
{
- values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ if (beentry->st_progress_command == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target);
+ else if (beentry->st_progress_command2 == cmdtype)
+ values[2] = ObjectIdGetDatum(beentry->st_progress_command_target2);
+
for (i = 0; i < PGSTAT_NUM_PROGRESS_PARAM; i++)
- values[i + 3] = Int64GetDatum(beentry->st_progress_param[i]);
+ values[i + 3] = Int64GetDatum(params[i]);
}
else
{
diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h
index 9aee8226347..d4c77f3df3f 100644
--- a/src/include/catalog/index.h
+++ b/src/include/catalog/index.h
@@ -45,9 +45,8 @@ typedef struct ReindexParams
/* flag bits for ReindexParams->flags */
#define REINDEXOPT_VERBOSE 0x01 /* print progress info */
-#define REINDEXOPT_REPORT_PROGRESS 0x02 /* report pgstat progress */
-#define REINDEXOPT_MISSING_OK 0x04 /* skip missing relations */
-#define REINDEXOPT_CONCURRENTLY 0x08 /* concurrent mode */
+#define REINDEXOPT_MISSING_OK 0x02 /* skip missing relations */
+#define REINDEXOPT_CONCURRENTLY 0x04 /* concurrent mode */
/* state info for validate_index bulkdelete callback */
typedef struct ValidateIndexState
@@ -71,7 +70,6 @@ extern void index_check_primary_key(Relation heapRel,
#define INDEX_CREATE_IF_NOT_EXISTS (1 << 4)
#define INDEX_CREATE_PARTITIONED (1 << 5)
#define INDEX_CREATE_INVALID (1 << 6)
-#define INDEX_CREATE_SUPPRESS_PROGRESS (1 << 7)
extern Oid index_create(Relation heapRelation,
const char *indexRelationName,
@@ -149,8 +147,7 @@ extern void index_build(Relation heapRelation,
Relation indexRelation,
IndexInfo *indexInfo,
bool isreindex,
- bool parallel,
- bool progress);
+ bool parallel);
extern void validate_index(Oid heapId, Oid indexId, Snapshot snapshot);
diff --git a/src/include/utils/backend_status.h b/src/include/utils/backend_status.h
index a334e096e4a..8b39c59df7b 100644
--- a/src/include/utils/backend_status.h
+++ b/src/include/utils/backend_status.h
@@ -169,6 +169,15 @@ typedef struct PgBackendStatus
Oid st_progress_command_target;
int64 st_progress_param[PGSTAT_NUM_PROGRESS_PARAM];
+ /*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
+
/* query identifier, optionally computed using post_parse_analyze_hook */
int64 st_query_id;
@@ -176,6 +185,16 @@ typedef struct PgBackendStatus
int64 st_plan_id;
} PgBackendStatus;
+/*
+ * Check the depth of progress tracking.
+ *
+ * 0 - tracking disabled
+ * 1 - enabled for the "main" command
+ * 2 - enabled for both main command and its sub-command.
+ */
+#define PGSTAT_PROGRESS_STATE(state) \
+ ((state)->st_progress_command == PROGRESS_COMMAND_INVALID ? 0 :\
+ (state)->st_progress_command2 == PROGRESS_COMMAND_INVALID ? 1 : 2)
/*
* Macros to load and store st_changecount with appropriate memory barriers.
--
2.52.0
--=-=-=--
^ permalink raw reply [nested|flat] 453+ messages in thread
* Allow progress tracking of sub-commands
@ 2026-06-29 11:57 Antonin Houska <[email protected]>
0 siblings, 1 reply; 453+ messages in thread
From: Antonin Houska @ 2026-06-29 11:57 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]
I've noticed several complaints about two commands trying to report their
progress at the same time, overwriting each other's output [1][2][3]. I posted
a prototype of an enhancement of the monitoring infrastructure that allows for
progress reporting of two commands w/o conflicts [4].
This is a new version that passes regression tests. I'll register it for the
upcoming CF.
[1] https://www.postgresql.org/message-id/5667.1774513434%40localhost
[2] https://www.postgresql.org/message-id/CALj2ACUgwSchK6jQ2CdKLBWUADTOE_zKdTff2Zg3E6hOuXKv-w%40mail.gma...
[3] https://www.postgresql.org/message-id/aj4gJQMba0kClQmj%40Mac
[4] https://www.postgresql.org/message-id/30939.1777888333%40localhost
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 453+ messages in thread
* Re: Allow progress tracking of sub-commands
@ 2026-06-29 16:03 Rahila Syed <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 453+ messages in thread
From: Rahila Syed @ 2026-06-29 16:03 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; [email protected]
Hi Antonin,
Thanks for suggesting enhancements to the progress reporting framework.
I have comments on the solution's approach.
/*
+ * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
+ * target can be different, e.g. when the sub-command builds an index on
+ * TOAST relation.
+ */
+ ProgressCommandType st_progress_command2;
+ Oid st_progress_command_target2;
+ int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
This approach only works for a nesting depth of 2, not for 3 or more
levels of subcommands, for instance.
I am not aware of a concrete example of such a command but I think we should
keep the design generic enough to accommodate this.
Shall we consider reporting the progress counters of a sub-command as
additional parameters
in the existing st_progress_param[] array? A top level command can
define the progress parameters
based on the number and type of sub-commands it expects to see. This
way we can even restrict or alter the counters we would like to report
for a sub-command. This also avoids changing PgBackendStatus every
time we encounter a command requiring deeper nesting levels.
Thank you,
Rahila Syed
On Mon, 29 Jun 2026 at 19:57, Antonin Houska <[email protected]> wrote:
>
> I've noticed several complaints about two commands trying to report their
> progress at the same time, overwriting each other's output [1][2][3]. I posted
> a prototype of an enhancement of the monitoring infrastructure that allows for
> progress reporting of two commands w/o conflicts [4].
>
> This is a new version that passes regression tests. I'll register it for the
> upcoming CF.
>
>
> [1] https://www.postgresql.org/message-id/5667.1774513434%40localhost
> [2] https://www.postgresql.org/message-id/CALj2ACUgwSchK6jQ2CdKLBWUADTOE_zKdTff2Zg3E6hOuXKv-w%40mail.gma...
> [3] https://www.postgresql.org/message-id/aj4gJQMba0kClQmj%40Mac
> [4] https://www.postgresql.org/message-id/30939.1777888333%40localhost
>
> --
> Antonin Houska
> Web: https://www.cybertec-postgresql.com
>
^ permalink raw reply [nested|flat] 453+ messages in thread
* Re: Allow progress tracking of sub-commands
@ 2026-07-01 15:24 Antonin Houska <[email protected]>
parent: Rahila Syed <[email protected]>
0 siblings, 1 reply; 453+ messages in thread
From: Antonin Houska @ 2026-07-01 15:24 UTC (permalink / raw)
To: Rahila Syed <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; [email protected]
Rahila Syed <[email protected]> wrote:
> Thanks for suggesting enhancements to the progress reporting framework.
>
> I have comments on the solution's approach.
Thanks.
> /*
> + * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
> + * target can be different, e.g. when the sub-command builds an index on
> + * TOAST relation.
> + */
> + ProgressCommandType st_progress_command2;
> + Oid st_progress_command_target2;
> + int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
>
> This approach only works for a nesting depth of 2, not for 3 or more
> levels of subcommands, for instance.
> I am not aware of a concrete example of such a command but I think we should
> keep the design generic enough to accommodate this.
I thought about all the current command types
typedef enum ProgressCommandType
{
PROGRESS_COMMAND_INVALID,
PROGRESS_COMMAND_VACUUM,
PROGRESS_COMMAND_ANALYZE,
PROGRESS_COMMAND_CREATE_INDEX,
PROGRESS_COMMAND_BASEBACKUP,
PROGRESS_COMMAND_COPY,
PROGRESS_COMMAND_REPACK,
} ProgressCommandType;
The typical problem occurs when REPACK performs reindexing (CREATE_INDEX). I
could only think of one case where the depth would be more than 2: an index
function (executed during the build) running another monitored command. In a
development build (with my patch applied), such a case would fire an assertion
in pgstat_progress_start_command(), while in a production build that innermost
command would only overwrite the status of the CREATE INDEX command. I don't
consider such a crazy case worth more effort.
> Shall we consider reporting the progress counters of a sub-command as
> additional parameters
> in the existing st_progress_param[] array? A top level command can
> define the progress parameters
> based on the number and type of sub-commands it expects to see. This
> way we can even restrict or alter the counters we would like to report
> for a sub-command. This also avoids changing PgBackendStatus every
> time we encounter a command requiring deeper nesting levels.
It might be possible to make sure in some cases that parameter numbers not
used by the main command are used by the sub-command, but I think it would
make coding quite tricky. And if you wanted to be sure that there are no
collisions of parameter numbers, you'd need st_progress_param[] to be twice as
large as the maximum number of parameters per command anyway. So this approach
wouldn't help as long as you're concerned of memory consumption.
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 453+ messages in thread
* Re: Allow progress tracking of sub-commands
@ 2026-07-14 16:50 Alberto Piai <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 1 reply; 453+ messages in thread
From: Alberto Piai @ 2026-07-14 16:50 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; Rahila Syed <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; [email protected]
Hi Antonin and Rahila,
thanks for working on this, as a user I find this a worthy improvement.
>> /*
>> + * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
>> + * target can be different, e.g. when the sub-command builds an index on
>> + * TOAST relation.
>> + */
>> + ProgressCommandType st_progress_command2;
>> + Oid st_progress_command_target2;
>> + int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
>>
>> This approach only works for a nesting depth of 2, not for 3 or more
>> levels of subcommands, for instance.
>> I am not aware of a concrete example of such a command but I think we should
>> keep the design generic enough to accommodate this.
...
> The typical problem occurs when REPACK performs reindexing (CREATE_INDEX). I
> could only think of one case where the depth would be more than 2: an index
> function (executed during the build) running another monitored command. In a
> development build (with my patch applied), such a case would fire an assertion
> in pgstat_progress_start_command(), while in a production build that innermost
> command would only overwrite the status of the CREATE INDEX command. I don't
> consider such a crazy case worth more effort.
I tend to agree that (for now, at least) a nesting level of 3 is an edge
case, but I have another concern with the current approach:
@@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
return;
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
- beentry->st_progress_command = cmdtype;
- beentry->st_progress_command_target = relid;
- MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
+ /* Sub-command should not be started w/o parent command. */
+ if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command = cmdtype;
+ beentry->st_progress_command_target = relid;
+ MemSet(&beentry->st_progress_param, 0,
+ sizeof(beentry->st_progress_param));
+ }
+ else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
+ {
+ Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
+
+ beentry->st_progress_command2 = cmdtype;
+ beentry->st_progress_command_target2 = relid;
+ MemSet(&beentry->st_progress_param2, 0,
+ sizeof(beentry->st_progress_param2));
+ }
+ else
+ {
+ /* Only one level of nesting is supported. */
+ Assert(false);
+ }
PGSTAT_END_WRITE_ACTIVITY(beentry);
}
The comments of the two macros warn about the fact that they create a
critical section, and code within the critical section should be kept as
simple as possible because any error would be promoted to PANIC. Now
sure, Assert() doesn't matter in production builds, but still I think
there is value in getting rid of a possible error path. (And a restart
to clean up shared memory would be pretty annoying even in debug
builds.)
Assuming the start()/end() calls are properly nested (which I'd assume
they are given the structure of the if/else conditions in start()/end()
in this patch), what about the following (names are all strawmen, as
this is just a suggestion):
- group command, target and params into a new struct, let's say Progress
- track the current level of nesting in the beentry, initially 0
- track an array of Progress structs in the beentry, of MAX_NESTING size
(2, for now)
Then ...start_command() would do something like the following:
PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
beentry->nesting++;
if beentry->nesting <= MAX_NESTING
beentry->progress[beentry->nesting-1].command = ...
...
PGSTAT_END_WRITE_ACTIVITY(beentry);
...end_command() and the update routines would have to reflect this of
course, and any reader would have to be changed. Readers would have to
go through an accessor that clamps the index to [0,MAX_NESTING) and make
this a noop otherwise.
One advantage of this would be that it makes the case of an unsupported
nesting level a noop, instead of an error.
The main advantage would be that the compiler would yell if we forgot to
change a reader.
While looking into this for example, I ran into some code in
vacuumlazy.c (heap_vacuum_rel()) which writes into st_progress_param
directly:
appendStringInfo(&buf, _("delay time: %.3f ms\n"),
(double) MyBEEntry->st_progress_param[PROGRESS_VACUUM_DELAY_TIME] / 1000000.0);
Purely theoretical of course, but if this was done by code which could
run both as a main command or a subcommand, this would end up writing
to the wrong spot.
Regarding the maximum level of nesting: right now we could say that
anything above 2 is not worth the bytes in MyBEEntry, and if we ever
encountered a case where it's worth reporting progress of a second
subcommand, this could be raised to 3.
What do you think, is something like this feasible? Maybe it would
address Rahila's point, while not being too much extra work / complex
code.
Regards,
Alberto
--
Alberto Piai
Sensational AG
Zürich, Switzerland
^ permalink raw reply [nested|flat] 453+ messages in thread
* Re: Allow progress tracking of sub-commands
@ 2026-07-22 08:28 Antonin Houska <[email protected]>
parent: Alberto Piai <[email protected]>
0 siblings, 1 reply; 453+ messages in thread
From: Antonin Houska @ 2026-07-22 08:28 UTC (permalink / raw)
To: Alberto Piai <[email protected]>; +Cc: Rahila Syed <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Alberto Piai <[email protected]> wrote:
> thanks for working on this, as a user I find this a worthy improvement.
>
> >> /*
> >> + * Some commands have a sub-command, e.g. REPACK (re)builds indexes. The
> >> + * target can be different, e.g. when the sub-command builds an index on
> >> + * TOAST relation.
> >> + */
> >> + ProgressCommandType st_progress_command2;
> >> + Oid st_progress_command_target2;
> >> + int64 st_progress_param2[PGSTAT_NUM_PROGRESS_PARAM];
> >>
> >> This approach only works for a nesting depth of 2, not for 3 or more
> >> levels of subcommands, for instance.
> >> I am not aware of a concrete example of such a command but I think we should
> >> keep the design generic enough to accommodate this.
> ...
> > The typical problem occurs when REPACK performs reindexing (CREATE_INDEX). I
> > could only think of one case where the depth would be more than 2: an index
> > function (executed during the build) running another monitored command. In a
> > development build (with my patch applied), such a case would fire an assertion
> > in pgstat_progress_start_command(), while in a production build that innermost
> > command would only overwrite the status of the CREATE INDEX command. I don't
> > consider such a crazy case worth more effort.
>
> I tend to agree that (for now, at least) a nesting level of 3 is an edge
> case, but I have another concern with the current approach:
>
> @@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
> return;
>
> PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
> - beentry->st_progress_command = cmdtype;
> - beentry->st_progress_command_target = relid;
> - MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
> + /* Sub-command should not be started w/o parent command. */
> + if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
> + {
> + Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
> +
> + beentry->st_progress_command = cmdtype;
> + beentry->st_progress_command_target = relid;
> + MemSet(&beentry->st_progress_param, 0,
> + sizeof(beentry->st_progress_param));
> + }
> + else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
> + {
> + Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
> +
> + beentry->st_progress_command2 = cmdtype;
> + beentry->st_progress_command_target2 = relid;
> + MemSet(&beentry->st_progress_param2, 0,
> + sizeof(beentry->st_progress_param2));
> + }
> + else
> + {
> + /* Only one level of nesting is supported. */
> + Assert(false);
> + }
> PGSTAT_END_WRITE_ACTIVITY(beentry);
> }
>
>
> The comments of the two macros warn about the fact that they create a
> critical section, and code within the critical section should be kept as
> simple as possible because any error would be promoted to PANIC. Now
> sure, Assert() doesn't matter in production builds, but still I think
> there is value in getting rid of a possible error path.
The patch only adds an if-else construct, but the other lines added are just
variants of the existing ones. So I don't think the patch increases the risk
of an error ERROR (promoted to PANIC).
> (And a restart to clean up shared memory would be pretty annoying even in
> debug builds.)
If Assert() fires, then what I consider annoying is the bug that caused it,
not the restart. Restart could be a problem if I wanted the other regression
tests to complete before fixing the bug. However that doesn't make much sense
to me because all the regression tests should be run again as soon as the bug
is fixed.
> Assuming the start()/end() calls are properly nested (which I'd assume
> they are given the structure of the if/else conditions in start()/end()
> in this patch), what about the following (names are all strawmen, as
> this is just a suggestion):
>
> - group command, target and params into a new struct, let's say Progress
> - track the current level of nesting in the beentry, initially 0
> - track an array of Progress structs in the beentry, of MAX_NESTING size
> (2, for now)
>
> Then ...start_command() would do something like the following:
>
> PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
> beentry->nesting++;
> if beentry->nesting <= MAX_NESTING
> beentry->progress[beentry->nesting-1].command = ...
> ...
> PGSTAT_END_WRITE_ACTIVITY(beentry);
>
> ...end_command() and the update routines would have to reflect this of
> course, and any reader would have to be changed. Readers would have to
> go through an accessor that clamps the index to [0,MAX_NESTING) and make
> this a noop otherwise.
Now that I compare my approach to yours, I'm not really opposed to this. The
fact that the coding is more generic doesn't imply that MAX_NESTING needs to
be very large.
> One advantage of this would be that it makes the case of an unsupported
> nesting level a noop, instead of an error.
I don't see an advantage exactly here: my patch does not raise ERROR, and I
think that Assert() is needed to check the nesting level even if it's
implemented in your way.
> The main advantage would be that the compiler would yell if we forgot to
> change a reader.
Not sure I understand the difference. Can you please describe the situation
w/o this advantage, i.e. w/o the compiler errors?
> While looking into this for example, I ran into some code in
> vacuumlazy.c (heap_vacuum_rel()) which writes into st_progress_param
> directly:
>
> appendStringInfo(&buf, _("delay time: %.3f ms\n"),
> (double) MyBEEntry->st_progress_param[PROGRESS_VACUUM_DELAY_TIME] / 1000000.0);
>
> Purely theoretical of course, but if this was done by code which could
> run both as a main command or a subcommand, this would end up writing
> to the wrong spot.
This looks like reading rather than writing, but I think I understand what you
mean.
> Regarding the maximum level of nesting: right now we could say that
> anything above 2 is not worth the bytes in MyBEEntry, and if we ever
> encountered a case where it's worth reporting progress of a second
> subcommand, this could be raised to 3.
Yes, we should be careful because MAX_NESTING items of the array would be
allocated for every single backend.
> What do you think, is something like this feasible? Maybe it would
> address Rahila's point, while not being too much extra work / complex
> code.
Yes, I think it's worth at least a prototype. Do you want to adjust my patch
yourself or should I do?
Thanks for your review anyway!
--
Antonin Houska
Web: https://www.cybertec-postgresql.com
^ permalink raw reply [nested|flat] 453+ messages in thread
* Re: Allow progress tracking of sub-commands
@ 2026-07-22 21:19 Alberto Piai <[email protected]>
parent: Antonin Houska <[email protected]>
0 siblings, 0 replies; 453+ messages in thread
From: Alberto Piai @ 2026-07-22 21:19 UTC (permalink / raw)
To: Antonin Houska <[email protected]>; Alberto Piai <[email protected]>; +Cc: Rahila Syed <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
On Wed Jul 22, 2026 at 10:28 AM CEST, Antonin Houska wrote:
>>
>> @@ -33,9 +37,30 @@ pgstat_progress_start_command(ProgressCommandType cmdtype, Oid relid)
>> return;
>>
>> PGSTAT_BEGIN_WRITE_ACTIVITY(beentry);
>> - beentry->st_progress_command = cmdtype;
>> - beentry->st_progress_command_target = relid;
>> - MemSet(&beentry->st_progress_param, 0, sizeof(beentry->st_progress_param));
>> + /* Sub-command should not be started w/o parent command. */
>> + if (beentry->st_progress_command == PROGRESS_COMMAND_INVALID)
>> + {
>> + Assert(beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID);
>> +
>> + beentry->st_progress_command = cmdtype;
>> + beentry->st_progress_command_target = relid;
>> + MemSet(&beentry->st_progress_param, 0,
>> + sizeof(beentry->st_progress_param));
>> + }
>> + else if (beentry->st_progress_command2 == PROGRESS_COMMAND_INVALID)
>> + {
>> + Assert(beentry->st_progress_command != PROGRESS_COMMAND_INVALID);
>> +
>> + beentry->st_progress_command2 = cmdtype;
>> + beentry->st_progress_command_target2 = relid;
>> + MemSet(&beentry->st_progress_param2, 0,
>> + sizeof(beentry->st_progress_param2));
>> + }
>> + else
>> + {
>> + /* Only one level of nesting is supported. */
>> + Assert(false);
>> + }
>> PGSTAT_END_WRITE_ACTIVITY(beentry);
>> }
>>
>>
>> The comments of the two macros warn about the fact that they create a
>> critical section, and code within the critical section should be kept as
>> simple as possible because any error would be promoted to PANIC. Now
>> sure, Assert() doesn't matter in production builds, but still I think
>> there is value in getting rid of a possible error path.
>
> The patch only adds an if-else construct, but the other lines added are just
> variants of the existing ones. So I don't think the patch increases the risk
> of an error ERROR (promoted to PANIC).
>
>> (And a restart to clean up shared memory would be pretty annoying even in
>> debug builds.)
>
> If Assert() fires, then what I consider annoying is the bug that caused it,
> not the restart. Restart could be a problem if I wanted the other regression
> tests to complete before fixing the bug. However that doesn't make much sense
> to me because all the regression tests should be run again as soon as the bug
> is fixed.
Sorry, I could have been clearer. I was referring specifically to the
Assert(), which would be hit in the edge case you mentioned (a command
trying to report progress past the 2nd level of nesting).
I think it would be nicer if attempts at progress reporting past the
maximum supported level of nesting would be a noop rather than an error,
and in general if the command didn't need to know whether it's reporting
progress at level 1 or level 2.
>> The main advantage would be that the compiler would yell if we forgot to
>> change a reader.
>
> Not sure I understand the difference. Can you please describe the situation
> w/o this advantage, i.e. w/o the compiler errors?
>
>> While looking into this for example, I ran into some code in
>> vacuumlazy.c (heap_vacuum_rel()) which writes into st_progress_param
>> directly:
>>
>> appendStringInfo(&buf, _("delay time: %.3f ms\n"),
>> (double) MyBEEntry->st_progress_param[PROGRESS_VACUUM_DELAY_TIME] / 1000000.0);
>>
>> Purely theoretical of course, but if this was done by code which could
>> run both as a main command or a subcommand, this would end up writing
>> to the wrong spot.
What I meant to say: with your patch, existing code referring to
st_progress_command would not break, but might find itself working with
st_progress_param, when it should be st_progress_param2 (depending on
how the command is nested).
If we instead changed to an array of structs, it would be a breaking
change and we would have to fix all the callers.
>
>> What do you think, is something like this feasible? Maybe it would
>> address Rahila's point, while not being too much extra work / complex
>> code.
>
> Yes, I think it's worth at least a prototype. Do you want to adjust my patch
> yourself or should I do?
Please go on, I was just trying to help as a reviewer. Looking forward
to the second version then :)
Regards,
Alberto
--
Alberto Piai
Sensational AG
Zürich, Switzerland
^ permalink raw reply [nested|flat] 453+ messages in thread
end of thread, other threads:[~2026-07-22 21:19 UTC | newest]
Thread overview: 453+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2023-09-25 05:01 [PATCH v8 5/7] Row pattern recognition patch (docs). Tatsuo Ishii <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-05-04 09:32 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:10 [PATCH] Allow progress tracking of sub-commands. Antonin Houska <[email protected]>
2026-06-29 11:57 Allow progress tracking of sub-commands Antonin Houska <[email protected]>
2026-06-29 16:03 ` Re: Allow progress tracking of sub-commands Rahila Syed <[email protected]>
2026-07-01 15:24 ` Re: Allow progress tracking of sub-commands Antonin Houska <[email protected]>
2026-07-14 16:50 ` Re: Allow progress tracking of sub-commands Alberto Piai <[email protected]>
2026-07-22 08:28 ` Re: Allow progress tracking of sub-commands Antonin Houska <[email protected]>
2026-07-22 21:19 ` Re: Allow progress tracking of sub-commands Alberto Piai <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox